Reshape and broadcast optimizations to avoid unnecessary data movement

ABSTRACT

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for transforming patterns of operations on tensors in a computational graph to reduce the memory burden incurred when reshape operations are performed, in particular when deployed to hardware platforms that have vector instructions or vector memory requiring alignment of operands.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of and claims the benefitof priority to U.S. application Ser. No. 16/402,981, filed on May 3,2019, the contents of which are hereby incorporated by reference.

BACKGROUND

The TensorFlow library can be used to create machine learning models,e.g., recurrent neural network (“RNN”) models, convolutional neuralnetwork (“CNN”) models, feedforward neural network models, and randomforest models. (TensorFlow is described in Abadi et al., TensorFlow: Asystem for Large-Scale Design and Implementation (OSDI '16), pp.265-283, Nov. 2-4, 2016. The software is available fromhttps://tensorflow.org.) The TensorFlow library can be used to representmachine learning models as TensorFlow graphs. Each node in a TensorFlowgraph represents an operation. Each edge in a TensorFlow graph isdirected and represents a flow of data that is into or out of the nodeinto which the edge is connected. The data is in the form of a tensor ofzero or more dimensions, in which each element has the same data type,e.g., 32-bit integer, double-length floating point, or string. A tensoris represented externally by vectors in paired brackets “[ ]”. Forexample, a one-dimensional (1D) tensor, also called a vector, of 3elements would be represented as [1, 2, 3]. A zero-dimensional tensor isa scalar. A two-dimensional (2D) tensor would be represented as [[1, 2,3], [4, 5, 6]]. The rank of this tensor, i.e., the number of dimensionsor the number of indices required to uniquely select each element of thetensor, is two. The shape of this tensor is [2, 3]. Two is the number ofelements in the zero-th dimension, namely, the two vectors (1D tensors)[1, 2, 3] and [4, 5, 6]; and three is the number of elements in thefirst dimension; that is, each of the vectors [1, 2, 3] and [4, 5, 6]have three elements. The shape of a tensor is itself a 1D tensor. As isconventional in many programming contexts, the numbering of dimensionsstarts with zero.

In this specification, the examples will be expressed using the PythonAPI for constructing and executing a TensorFlow graph. The TensorFlowmodule can be loaded thus:

-   -   import tensorflow as tf

The TensorFlow operations include shape, reshape, broadcast, and reduceoperations. These will be described below, omitting parameters andaspects from the description that are not material to thisspecification.

When executed, the shape operation returns the shape, i.e., thedimensions, of an input tensor as a 1D tensor. In the following example:

-   -   X=tf.constant([[[1, 1, 1], [2, 2, 2]], [[3, 3, 3], [4, 4, 4]]])        tf.shape(X)        the shape operations returns the tensor [2, 2, 3], which        represents the dimensions of tensor X.

When executed, the reshape operation returns a tensor having the sameelement values in the same order as the input tensor, but with a shapethat is defined by a shape tensor input. In the following example,

-   -   X=tf.constant([[[1, 1], [2, 2]], [[3, 3], [4, 4]]])        tf.reshape(X, [2, 4])        the reshape operation receives tensor X and a one-dimensional        tensor [2, 4] representing a desired shape as input parameters.        The reshape operation returns a tensor [[1, 1, 2, 2], [3, 3, 4,        4]] that has the same elements as the input tensor X and has the        desired shape, namely [2, 4]. The desired shape input to a        reshape operation can have more or fewer dimensions than the        input tensor has.

The broadcast operations include broadcast to. Broadcasting is theprocess of making arrays that have compatible shapes for arithmeticoperations. Two shapes are compatible if for each correspondingdimension pair of their shapes, the dimensions are either equal or oneof them is one. When a tensor is broadcast to a shape, the operationstarts with the trailing dimensions and works its way forward.

Thus, when executed, the broadcast to operation returns a tensor that isthe input tensor replicated as many times as need until the requested,specified shape is reached. In the following example:

-   -   V=tf.constant([7, 8])    -   tf.broadcast to(V, [2, 3])        the broadcast to operation receives as input a tensor V and a        tensor [2, 3] specifying the desired shape. It returns a tensor        [[7, 7, 7], [8, 8, 8]], which has the desired shape.

The reduce operations include reduce_all, reduce_any, reduce_sum, andreduce_mean. The reduce operations return an output tensor thatgenerally has a smaller rank and a smaller number of elements than theinput tensor.

The reduce operations receive an input tensor and an axis tensor. Theelements of the axis tensor identify dimensions of the shape of theinput tensor. The reduce operations reduce the input tensor along thedimensions specified by the axis tensor. For example, in

-   -   X=tf.constant([[1, 1, 1], [1, 1, 1]])        the shape of X is [2, 3], i.e., X is a tensor with two rows and        three columns:

1 1 1 1 1 1

To take the specific reduce operation reduce_sum as an example, with anaxis tensor [0], which identifies the rows, i.e., the zero-th dimensionof X, the operation

-   -   tf.reduce_sum(x, [0])        when executed reduces the tensor along zero-th dimension (the        rows) and sums the rows [1, 1, 1]+[1, 1, 1] to return [2, 2, 2].        When    -   tf.reduce_sum(x, [1])        is executed, the reduction is along the first dimension (the        columns) and sums the columns [1, 1]+[1, 1], +[1, 1] to return        [3, 3]. When    -   tf.reduce_sum(x, [0, 1])        is executed, the reduction is along both dimensions and sums to        return the scalar (0D tensor) 6.

The shape of a tensor returned by a reduce operation has the dimensionsof the input tensor without the indices specified by the axis tensor.

Other reduce operations return tensors whose elements' values arecomputed by other operations. For example, the reduce_all operationcomputes the logical AND, the reduce_any operation computes the logicalOR, the reduce_mean operation computes the mean, and so on.

In some scenarios, a user uses a compiler, e.g., a Just-in-Time (“JIT”)compiler, to compile TensorFlow graphs to graphs to input into an XLAcompiler. (A JIT compiler is described inhttps://www.tensorflow.org/xla/jit). The input language to XLA is called“HLO IR”, or just HLO (High Level Optimizer). The XLA compiler takesgraphs, i.e., computations, defined in HLO and compiles them intomachine instructions for various architectures, performingtarget-dependent optimizations and generating target-dependent code.

The nodes in an HLO graph represents operations. Each edge in the graphis directed and represents a flow of data that is into or out of thenode to which the edge is connected. This data is in the form of atensor. The operations represented in an HLO graph correspond tooperations in a TensorFlow flow from which the HLO graph was generated.In particular, an HLO graph can include reshape, reduce, and broadcastoperations

The binaries generated by the XLA compiler are deployed onto hardwareand executed by the specific processors of the hardware. Some processorsimplement instructions that operate on vectors. In order for a processorto perform vector instructions that operate on tensor data, the tensorsmust be stored such that each of the tensor's vectors that will beoperated on by vector instructions are aligned on a vector boundary, asspecified for the processor.

For example, if a reshape operation receives a tensor [1, 2, 3, 4, 5, 6,7, 8, 9] and a tensor specifying a shape [3, 3] as input parameters, theresulting tensor [[1, 2, 3], [4, 5, 6], [7, 8, 9]] has three vectors [1,2, 3], [4, 5, 6], and [7, 8, 9] that must be moved so that they arealigned on a vector boundary if the vectors do not happen to be alignedon a vector boundary as required by the specific processor of thehardware.

SUMMARY

This specification describes optimization techniques that can beimplemented in an XLA compiler to reduce the memory burden of particularsequences of operations that include a reshape operation.

These optimizations are particularly helpful in operation sequences thatimplement machine learning techniques like group normalization(https://arxiv.org/pdf/1803.08494.pdf) and ghost batch normalization(https://arxiv.org/pdf/1705.08741.pdf). In a direct implementation ofsuch machine learning techniques, the input tensor is reshaped to ahigher number of dimensions. Then a reduction is performed across somedimensions changed by the reshape and other dimensions not changed bythe reshape. The derivative of the reduce and reshape is a broadcast andthat is reshaped back to the original shape. The broadcast is also madeinto some dimensions changed by the reshape and some dimensions notchanged by the reshape. The reshape operations do nothing to the data onplatforms with a linear address space. However, on platforms with avector memory, reshapes will in general change shape alignment relativeto the vector memory, requiring data to be moved. This specificationdescribes optimizations that reduce the sizes of tensors that needs tobe moved to achieve alignment with respect to a vector memory.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart illustrating an example process that implementscompiler transformations that detect a pattern of operations thatinclude reshape operations that can be transformed to minimize the sizeof the reshape operations.

FIG. 2 is a flowchart illustrating an example process of transformation.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

FIG. 1 is a flowchart illustrating an example process 100 thatimplements compiler transformations that detect a pattern of operationsthat include reshape operations that can be transformed to minimize thesize of the reshape operations. The compiler transformations will bedescribed with reference to an XLA compiler and to patterns of reshape,reduce, and broadcast operations in a computation graph. The compilerand the process can be implemented on and performed by a system of oneor more computers in one or more locations.

The process detects a pattern of operations that include reshapeoperations that can be transformed (102). This pattern can be a patternof operations in an XLA graph. One such pattern is red uce(reshape(X))when one or more dimensions are reduced that are not changed in thereshape. In this pattern, a reshape operation is performed on an inputtensor X and returns a tensor that is the input to a reduce operation.

The process transforms the operations to a pattern of operations with asmaller size of reshapes (104). For example, operations fitting thereduce(reshape(X)) pattern mentioned above are transformed to areduce(reshape(reduce(X))) pattern of operations. This transformationincreases the amount of compute by adding an additional reduce; but ithas the important advantage of reducing the total size of reshapes,since both computations have one reshape and the latter reshape has haddimensions reduced and as a result is strictly smaller in number ofelements than the original. The process continues to detect andtransform until no further patterns are detected, at which point thecompiler generates code that is specific to target hardware and thatimplements computations including the transformed patterns (106). Thenthe compiler, or other element of a TensorFlow infrastructure, deploysthe generated code to the target hardware for execution (108).

An example broadcast version of another pattern that can be optimized isreshape(add(reshape(X), broadcast(Y)) where the input shape of tensor Xis the same as the output shape of the tensor returned by the outermostreshape. This pattern of operations is transformed to add(X,broadcast(reshape(broadcast(Y)))), which has both fewer reshapes intotal and has a reshape of smaller size, since the output of the reshapeis broadcasted into a larger shape because it has extra dimensions.

This transformation can apply to an arbitrary subcomputation thatmatches the following pattern:

reshape(  f(    g(reshape(G) ) ,    h(reshape(H) ) , . . ,   a(broadcast(A) ) ,    b(broadcast(B)   )  ),which is transformed by the optimization to

f(   g(G) ,   h(H) , . . . ,   a( broadcast( reshape(broadcast(A) ) ) ,  b( broadcast( reshape(broadcast(B) ) )  )The lower-case letters f, g, h, a, and b represent mathematicaloperations in the graph. The pattern for his transformation can be foundusing a depth-first method of searching the graph. In someimplementations, for simplicity, the search is in post order, i.e.,topological sort with producers before consumers and transform the graphin place. When the search finds a matching subtree, it replicates thesubtree and replaces the users of the original subtree root with the newsubtree root. Other compiler passes fix up replicated and dead code.Other methods of searching the graph for patterns in a computation graphcan also be used.

With these two pattern transformations—from the forms reduce (reshape(X)) and reshape(f(reshape(X), broadcast(Y))—group normalization andvirtual batch normalization and their derivatives can be done withsmaller reshapes and the resulting smaller memory requirements.

For example, an implementation of the group norm is naturally expressedas

-   -   reduce(reshape(image, [B,H,W,C/G,G]), [1,2,3]).

The shape of the image input, above, is [B,H,W,C], with dimensions forthe batch size of the batch of images in the input and the height,width, and channels of the images. The number of groups is G groups. Ifthis expression is executed in this form, it creates a largeintermediate tensor on certain hardware platforms and a slow reshape.The above transformations improve the computation by transforming it tothe following form:

-   -   reduce(reshape(reduce(image, [1,2]), [B,C/G,G]), [1]),        which, for purposes of describing the process of transformation        will be represented as:    -   Y=reshape(X, [B, H, W, C/G, G])    -   Z=reduce(Y, [1, 2, 3])

The reduce operation reduces tensor Y shaped [3, H, W, C/G, G] on thedimensions specified by the axis tensor [1, 2, 3] and returns a tensorZ. The axis tensor [1, 2, 3] represents the dimensions [H, W, C/G] oftensor Y along which the reduce operation reduces. The reduce operationreturns a tensor Z of shape [B, G].

How the parameters of the original pattern are mapped to appropriateinputs to the transformation pattern is described below.

The reduce(reshape(reduce(X, [1, 2]), [B, C/G, G]), [1]) transformationwill be represented for the purpose of discussion as:

-   -   W=reduce(X, [1, 2])    -   Y2=reshape(W, [B, C/G, G])    -   Z2=reduce (Y₂, [1])

FIG. 2 is a flowchart illustrating an example process 200 oftransformation. This will be described in reference to the examplepattern just described. This process 200 is an example implementation ofa transformation (104) described above in reference to FIG. 1 .

The process determines the final output dimensions of a tensor returnedby the original pattern of operations (202). The final output dimensionsare determined by comparing the shape of X to the final output shape. Inthe example, the original pattern example receives a tensor X shaped [B,H, W, C] and returns a tensor shaped [B, G].

The process reduces along dimensions of the input tensor that areneither in the final output nor affect the reshape by the reshapeoperation of the original pattern (202). In the example, from theoriginal pattern

-   -   Y=reshape(X, [B, H, W, C/G, G])    -   Z=reduce(Y, [1, 2, 3])        the compiler determines from axis tensor input to the reshape        operation that the zero-th and third index, i.e., B and C, of        tensor X are in the final output and affect the final output,        respectively. Therefore, the X is reduced along the first and        second dimensions of X, i.e., the H and W dimensions:    -   W=Reduce(X, [1, 2])

The process reshapes the output tensor of the reduce operation (206).The output tensor of the reduce operation, W in the example, is reshapedto the shape of the original pattern but without the dimensions that arenot in the final output or transformed. In the original pattern, thethird dimension, i.e., C is divided by G, and a fourth dimension, i.e.,G is added. The zeroth dimension, i.e., B, is in the final output.Therefore, the reshape operation in the transformation reshapes thereduced tensor to [B, C/G, G]:

-   -   Y2=reshape(W, [B, C/G, G])

The process reduces the output tensor of the reshape operation along anydimensions that are not in the output tensor of the original pattern(208). In the example, the original pattern outputs a tensor shaped [B,G]. Therefore, the first index of the output of reshape operation in thetransformation is reduced and the reduce operation returns a tensorshaped [B, G]:

-   -   Z2=reduce(Y₂, [1])

The same rules are applied to transform original patterns of the formreshape(operator(reshape(X), broadcast(Y)) into the form operator(X,broadcast(reshape(broadcast(Y)))).

Embodiments of the subject matter and the actions and operationsdescribed in this specification can be implemented in digital electroniccircuitry, in tangibly-embodied computer software or firmware, incomputer hardware, including the structures disclosed in thisspecification and their structural equivalents, or in combinations ofone or more of them. Embodiments of the subject matter described in thisspecification can be implemented as one or more computer programs, e.g.,one or more modules of computer program instructions, encoded on acomputer program carrier, for execution by, or to control the operationof, data processing apparatus. The carrier may be a tangiblenon-transitory computer storage medium. Alternatively or in addition,the carrier may be an artificially-generated propagated signal, e.g., amachine-generated electrical, optical, or electromagnetic signal, thatis generated to encode information for transmission to suitable receiverapparatus for execution by a data processing apparatus. The computerstorage medium can be or be part of a machine-readable storage device, amachine-readable storage substrate, a random or serial access memorydevice, or a combination of one or more of them. A computer storagemedium is not a propagated signal.

The term “data processing apparatus” encompasses all kinds of apparatus,devices, and machines for processing data, including by way of example aprogrammable processor, a computer, or multiple processors or computers.Data processing apparatus can include special-purpose logic circuitry,e.g., an FPGA (field programmable gate array), an ASIC(application-specific integrated circuit), or a GPU (graphics processingunit). The apparatus can also include, in addition to hardware, codethat creates an execution environment for computer programs, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them.

A computer program, which may also be referred to or described as aprogram, software, a software application, an app, a module, a softwaremodule, an engine, a script, or code, can be written in any form ofprogramming language, including compiled or interpreted languages, ordeclarative or procedural languages; and it can be deployed in any form,including as a stand-alone program or as a module, component, engine,subroutine, or other unit suitable for executing in a computingenvironment, which environment may include one or more computersinterconnected by a data communication network in one or more locations.

A computer program may, but need not, correspond to a file in a filesystem. A computer program can be stored in a portion of a file thatholds other programs or data, e.g., one or more scripts stored in amarkup language document, in a single file dedicated to the program inquestion, or in multiple coordinated files, e.g., files that store oneor more modules, sub-programs, or portions of code.

The processes and logic flows described in this specification can beperformed by one or more computers executing one or more computerprograms to perform operations by operating on input data and generatingoutput. The processes and logic flows can also be performed byspecial-purpose logic circuitry, e.g., an FPGA, an ASIC, or a GPU, or bya combination of special-purpose logic circuitry and one or moreprogrammed computers.

Computers suitable for the execution of a computer program can be basedon general or special-purpose microprocessors or both, or any other kindof central processing unit. Generally, a central processing unit willreceive instructions and data from a read-only memory or a random accessmemory or both. The essential elements of a computer are a centralprocessing unit for executing instructions and one or more memorydevices for storing instructions and data. The central processing unitand the memory can be supplemented by, or incorporated in,special-purpose logic circuitry.

Generally, a computer will also include, or be operatively coupled toreceive data from or transfer data to one or more mass storage devices.The mass storage devices can be, for example, magnetic, magneto-optical,or optical disks, or solid state drives. However, a computer need nothave such devices. Moreover, a computer can be embedded in anotherdevice, e.g., a mobile telephone, a personal digital assistant (PDA), amobile audio or video player, a game console, a Global PositioningSystem (GPS) receiver, or a portable storage device, e.g., a universalserial bus (USB) flash drive, to name just a few.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on, orconfigured to communicate with, a computer having a display device,e.g., a LCD (liquid crystal display) monitor, for displaying informationto the user, and an input device by which the user can provide input tothe computer, e.g., a keyboard and a pointing device, e.g., a mouse, atrackball or touchpad. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback, e.g., visual feedback,auditory feedback, or tactile feedback; and input from the user can bereceived in any form, including acoustic, speech, or tactile input. Inaddition, a computer can interact with a user by sending documents toand receiving documents from a device that is used by the user; forexample, by sending web pages to a web browser on a user's device inresponse to requests received from the web browser, or by interactingwith an app running on a user device, e.g., a smartphone or electronictablet. Also, a computer can interact with a user by sending textmessages or other forms of message to a personal device, e.g., asmartphone that is running a messaging application, and receivingresponsive messages from the user in return.

This specification uses the term “configured to” in connection withsystems, apparatus, and computer program components. That a system ofone or more computers is configured to perform particular operations oractions means that the system has installed on it software, firmware,hardware, or a combination of them that in operation cause the system toperform the operations or actions. That one or more computer programs isconfigured to perform particular operations or actions means that theone or more programs include instructions that, when executed by dataprocessing apparatus, cause the apparatus to perform the operations oractions. That special-purpose logic circuitry is configured to performparticular operations or actions means that the circuitry has electroniclogic that performs the operations or actions.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of what isbeing claimed, which is defined by the claims themselves, but rather asdescriptions of features that may be specific to particular embodimentsof particular inventions. Certain features that are described in thisspecification in the context of separate embodiments can also beimplemented in combination in a single embodiment. Conversely, variousfeatures that are described in the context of a single embodiment canalso be implemented in multiple embodiments separately or in anysuitable subcombination. Moreover, although features may be describedabove as acting in certain combinations and even initially be claimed assuch, one or more features from a claimed combination can in some casesbe excised from the combination, and the claim may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings and recited inthe claims in a particular order, this should not be understood asrequiring that such operations be performed in the particular ordershown or in sequential order, or that all illustrated operations beperformed, to achieve desirable results. In certain circumstances,multitasking and parallel processing may be advantageous. Moreover, theseparation of various system modules and components in the embodimentsdescribed above should not be understood as requiring such separation inall embodiments, and it should be understood that the described programcomponents and systems can generally be integrated together in a singlesoftware product or packaged into multiple software products.

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. For example,the actions recited in the claims can be performed in a different orderand still achieve desirable results. As one example, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In some cases, multitasking and parallel processing may beadvantageous.

What is claimed is:
 1. A method performed by one or more computers, themethod comprising: detecting an original pattern of operations in agraph of computational operations on tensors, wherein the originalpattern of operations returns a final output tensor and takes as inputan input tensor, wherein the original pattern includes an originalreshape operation that (i) returns an original tensor and (ii) can betransformed to use less memory; transforming the original pattern ofoperations to a new pattern of operations with one or more reshapeoperations that return tensors smaller than the original tensor; andgenerating executable code that is specific to a target hardwareplatform and that implements computations represented by the new patternof operations.
 2. The method of claim 1, further comprising: deployingthe generated code to the target hardware platform for execution.
 3. Themethod of claim 1, wherein: the original reshape pattern of operationsrequires data to be moved to satisfy alignment requirements for vectorinstructions or vector memory on the target hardware platform.
 4. Themethod of claim 1, wherein transforming the original pattern ofoperations comprises: determining final output dimensions of the finaloutput tensor returned by the original pattern of operations; reducingalong dimensions of the input tensor that are neither in the finaloutput tensor nor affect the reshape by the original reshape operationto return a first intermediate-result tensor; reshaping the firstintermediate-result tensor to return a second intermediate-resulttensor; and reducing the second intermediate-result tensor along anydimensions that are not in dimensions of the final output tensor fromthe original pattern of operations.
 5. One or more non-transitorycomputer-readable storage media encoded with instructions that, whenexecuted by one or more computers, cause the one or more computers toperform actions comprising: detecting an original pattern of operationsin a graph of computational operations on tensors, wherein the originalpattern of operations returns a final output tensor and takes as inputan input tensor, wherein the original pattern includes an originalreshape operation that (i) returns an original tensor and (ii) can betransformed to use less memory; transforming the original pattern ofoperations to a new pattern of operations with one or more reshapeoperations that return tensors smaller than the original tensor; andgenerating executable code that is specific to a target hardwareplatform and that implements computations represented by the new patternof operations.
 6. The method of claim 5, wherein the actions furthercomprise: deploying the generated code to the target hardware platformfor execution.
 7. The method of claim 5, wherein: the original reshapepattern of operations requires data to be moved to satisfy alignmentrequirements for vector instructions or vector memory on the targethardware platform.
 8. The method of claim 5, wherein transforming theoriginal pattern of operations comprises: determining final outputdimensions of the final output tensor returned by the original patternof operations; reducing along dimensions of the input tensor that areneither in the final output tensor nor affect the reshape by theoriginal reshape operation to return a first intermediate-result tensor;reshaping the first intermediate-result tensor to return a secondintermediate-result tensor; and reducing the second intermediate-resulttensor along any dimensions that are not in dimensions of the finaloutput tensor from the original pattern of operations.
 9. A systemcomprising: one or more computers and one or more storage devices onwhich are stored instructions that are operable, when executed by theone or more computers, to cause the one or more computers to performactions comprising: detecting an original pattern of operations in agraph of computational operations on tensors, wherein the originalpattern of operations returns a final output tensor and takes as inputan input tensor, wherein the original pattern includes an originalreshape operation that (i) returns an original tensor and (ii) can betransformed to use less memory; transforming the original pattern ofoperations to a new pattern of operations with one or more reshapeoperations that return tensors smaller than the original tensor; andgenerating executable code that is specific to a target hardwareplatform and that implements computations represented by the new patternof operations.
 10. The method of claim 9, wherein the actions furthercomprise: deploying the generated code to the target hardware platformfor execution.
 11. The method of claim 9, wherein: the original reshapepattern of operations requires data to be moved to satisfy alignmentrequirements for vector instructions or vector memory on the targethardware platform.
 12. The method of claim 9, wherein transforming theoriginal pattern of operations comprises: determining final outputdimensions of the final output tensor returned by the original patternof operations; reducing along dimensions of the input tensor that areneither in the final output tensor nor affect the reshape by theoriginal reshape operation to return a first intermediate-result tensor;reshaping the first intermediate-result tensor to return a secondintermediate-result tensor; and reducing the second intermediate-resulttensor along any dimensions that are not in dimensions of the finaloutput tensor from the original pattern of operations.